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Abstract 

Recently, Cahill and Mixon completely characterized the sensing operators in many com¬ 
pressed sensing instances with a robust width property. The proposed property allows uni¬ 
formly stable and robust reconstruction of certain solutions from an underdetermined linear 
system via convex optimization. However, their theory does not cover the Lasso and Dantzig 
selector models, both of which are popular alternatives in the statistics community. In this 
letter, we show that the robust width property can be perfectly applied to these two models 
as well. Our results solve an open problem left by Cahill and Mixon. 
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1. Introduction 


One of the main assignments of compressed sensing is to understand when it is possible 
to recover structured solutions to underdetermined systems of linear equations [l| . During 
the past decade, there have developed many reconstruction guarantees; well-known concepts 
include restricted isometry property, null space property, coherence property, dual certihcate, 
and more (the interested readers could refer to j3, 0, ^). However, none of them is proved 
necessary for uniformly stable and robust reconstruction. Recently, Cahill and Mixon in 
introduced a new notion-robust width property, which completely characterizes the sensing 
operators in many compressed sensing instances. They restricted their attention into the 
following constrained optimization problem; 


min ||a;||jj, subject to ||<I)a; — y \\2 < e (Q^) 

such that their theory can not cover the Lasso and Dantzig selector models, both of which are 
popular alternatives in the statistics community. Here, || • ||(j is some norm used to promote 
certain structured solutions, operator $ and data y are given, and e measures the error. In 
this letter, we extend their results to two other probably more popular optimization problems 
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of the Lasso/Basis Pursuit and Dantzig selector types. Our derived results completely solve 
an open problem left by Cahill and Mixon and hence prove that the notion of robust width 
is indeed a ubiquitous property. In the following, we recall some notations appeared in the 
paper [^. 

Let be some unknown member of a finite-dimensional Hilbert space "H, and let $ : 
—)■ denote some known linear operator, where F is either M or C. Subset M C is a 

particular subset that consists of some type of structured members. is the unit |j-ball. 


2. Robust width 

The robust width property was formally proposed in j^. We write down the definition 
and its equivalent form as follows. 

Definition 1. (^) We say a linear operator $ ; "H —)■ F^ satisfies the {p, a)-robust width 
property over if 

lla^lh < p\\x\\i 

for every x G "H such that ||‘hx ||2 < q;||x|| 2 ; or equivalently if 

||* hx ||2 > q ;|| x ||2 


for every x G "H such that ||x ||2 > p||2:|ltt- 


Here, we would like to point out the definition above is not completely new. In fact, when 
restricted to the case of £i-minimization, it reduces to the £i-constrained minimal singular 
value property which was originally dehned in 


Definition 2. For any k G {1,2, • • • ,iV} and matrix <h G define the ii-constrained 

minimal singular value of <h by 


rk{^) 


min 

xj^0,xGSk 


||$ X ||2 



2 


where S'a, = (x G : ||x||i < \/^||x|| 2 }. //> 0, then we say $ satisfies the ii- 
constrained minimal singular value property with rfe(<h). 

Work exploited the geometrical aspect of the £i-constrained minimal singular value 
property. 


3. Main results 

We hrst introduce the definition of compressed sensing space. 
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Definition 3. (W A compressed sensing space (71, A, || • ||[() with hound L consists of a 
finite-dimensional Hilbert space H, a subset A ffH, and a norm || • ||jj on H with following 
properties: 

(i) 0 e 

(a) For every a & A and v ^FL, there exists a decomposition v = zi + Z 2 such that 

Ik + = Iklitt + Ikilk Ikslitt < l\\v\\2- 

The sub differential df{x) of a convex function / at a: is the set-valued operator given 
by 

df{x) = {ueFL: f{y) > f{x) + {u,y-x)yy E H}. 

The following lemma will be useful to establish our main results. 

Lemma 1. Let || • ||o be the dual norm of || • ||n on Ft. If u E 9||x||[j, then ||m|1o <1. If x 0 
and u E 5||a:||[j, then ||n||o = 1. 

Proof. From the convexity of || ■ |ltt sub differential definition, for any u E 5||x||[j and 

V eFL it holds 

Iklitt > Iklitt + {u,v - x). 

Set V = 0 and n = 2a; to get {u,x) > ||x||[t and {u,x) < ||x|l[j respectively. This implies 
{u,x) = ||x||[( and hence {u,v) < ||n||j. Similarly, by taking —v E FL, we can get —{u,v) < 
||n||(j. Thus, |(n,n)| < ||n||ij. Therefore, 

||n||o = sup |{M,n)|< sup ||n||[( < 1. 

Ihlltt<i lhlltt<i 

When X 7 ^ 0, by the Cauchy-Schwartz inequality we get that ||x||[t = {u,x) < ||a;||(j||n||o and 
hence ||m||^ > 1. So it must have ||n||o = 1. □ 

Now, we state the characterization of uniformly stable and robust reconstruction via the 
Lasso/Basis Pursuit type model by utilizing the (p, Q!)-robust width property. 

Theorem 1. For any CS space {Ft, A, || • Hj) with bound L and any linear operator $:?{—)■ 
F^, the following are eguivalent up to constants: 

(a) <F satisfies the {p,a)-robust width property over 

(b) For every x'^ E FL, k E (0,1), A > 0 and u E satisfying ||<F^a;|lo < kX, any solution 
X* to the unconstrained optimization model 

min^||$x - (<Fx^-Fn;)|| 2 -h A||x|ltj (Pa) 

satisfies ||x* — x ^\\2 < Co||x^ — a||jj -|- Ci • A for every a E A. 
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In particular, (a) implies (b) with 




-1 


2p 

provided p < Also, (b) implies (a) with 

p = 2Co, a = 


a^p 




2tCi 


where r = sup||^||^<i ||$x|| 2 . 

Proof. Let z = x* — x^. We divide the proof of (a) ^ {h) into four steps. They are partially 
inspired by Q and j^. 

Step 1: Prove the first relationship: 


\x 


n\\z\U < \\x^ 


( 1 ) 


Since x* is a minimizer to (|PaD i we have 
1 


Hence, 


- {^x^ + w)\\l + A||a;*|lt( < - {^x^ + w)!!^ + A|lx^| 


^||($a:* - d>x^) - w\\l + A||x*||s < ^||u;||^ + A||a:^| 


Rearrange terms to give 

A||a:*||t) < -^||<h(a;* - x^)\\l + ($(a;* - x^),w) + A||a;^||jj < {x* - x^, AA'w) + A||x‘’||tt. 

By the Cauchy-Schwartz inequality and the condition < kA, we obtain that 

{x* — x^, ^'^w) < llx* — x^||jj||$^t(;|lo < kX\\x* — x^lljj. 

Thus, A||a:*||o < KA||a:* — x^Hjj + A||a:*’||o from which the first relationship follows. 

Step 2: Prove the second relationship: 

2 9T 

Ikll# < ^+ ^-Ikih- (2) 

Pick a G A, and decompose z = x* — x'^ = Zi + Z 2 according to the property (ii) in Definition 
[3] so that ||a + = ||a||[( + ||zi||o and \\z 2 \\^ < L||z|| 2 . In light of ([I]), we derive that 


a h + - 


«ll# > 

> |lx*|h - K\\z\\i 
= \\x^ + {x* — x^ 


IZ" 'T* '10'^ 

tV I cL oL 


= \\a + - a) + zi + Z2|ltt - «I|d + ^2||tt 

> ||a + zilh - ||x^ - a|h - (1 + «)|ki|h - 

= l|a|h + Ikilh - - all# - (1 + K)||2:2||fl - k \\ zi \\^ 

= l|a||# + (1 - «:)||d||# - Ik^ - all# - (1 + «:)||z 2 ||#- 
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Rearrange terms to give 


which implies 

2 2 

Iklb < Ikillfl + Ik 2 ||fl < ^- -\\x^ - alls + --11^211#. 

J. — rC J. — Ki 

Thus, the second relationship follows by invoking 11^211(1 < -^Iklh- 
Step 3 : Derive the upper bound: 

\\^z\\l < (1 + «:)A|1^||#. 


The optimality condition of {\Px\j reads 

+ w — $x*) G A ■ (9||x*||(j. 
By using Lemma [H we get + w — $x*)||o < A. Thus, 


< — tc)|lo + 

^ A T kX = (1 T KjX. 


(3) 


Therefore, 

\\<^z\\l = {z, < ll^lls ■ ||<h^<h 2 ;||o < (1 + k)A| 12 ;||s, 

where the hrst inequality follows from the Cauchy-Schwartz inequality. 

Step 4: Finish the proof. Assume || 2:||2 > Cq ■ \\x^ — a||(j, since otherwise we are done. In 
light of ([2]), we obtain 


D « < 


2L 


Lc'o(i 


K 


K 


- 1 | 


D ||2 = P IDII 2 


i.e., || 2;||2 > p||-2||tt- By the (p, a)-robust width property of $, we have ||‘h. 2||2 > ctikih- 
Utilizing the upper bound of ll'h^lli in Step 3, we derive that 


aW, < ||fc||= < (1 + k)A||z||, < 1L!^||j||2. 

Thus, 

Ikih < = Cl • A < Co\\x^ - a|lp + Cl • A. 

a^p 

This completes the proof of (a) ^ {b). 
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The proof of (fo) ^ (a). Pick such that ||$x *’||2 < a||x^|| 2 . By the expression of 
r = sup||j,|||j<;^ IIand using the Cauchy-Schwartz inequality, we derive that 


T • a||x ^||2 > r • ||<hx ^||2 = sup ||<hx ||2 • ||‘hx ^||2 

lhlltt<i 

> sup ($x, <hx^)= sup (x, <h^<hx*’) 
lhllit<i lhllit<i 

= ||$^$X^||o. 


Let \ = K ^ra||x ^||2 and oj = —<f)xk Then, we have 

kX = T ■ a||x^||2 > ||$^d)x^||o = ||<h^tc 


which implies that the choosing of A and u satisfies the constrained condition ||$^t(;||o < kX. 
Thereby, we can take u = —<f)x^ and hence conclude that x* = 0 is a minimizer of 
Thus, 

< C'o||a:*’||tt + CiX = CoUx^lljj + CiK“Va||x^||2. 


Take a = 


2tCi 


xq |2 = ||x 

and p = 2 Cq and rearrange terms to give 

Co 


P II 2 < 


1 — CiK ^ra 

So the (p, Q;)-robust width property of <h holds. 


3 ^ 11 # = p\\xi\i- 


□ 


0 /. 


to obtain a corresponding result for (Qe), it suffices for 


Remark 1. In the paper 
to satisfy: 

r^jll^lltt — l|0|ltt every x G "H, and 

(ii) ||x + y\\i < ||x||j + \\y\\i for every x, p G 'H. 

In contrast, Theorem 1 not only requires (i) and (ii) above, but also utilizes the convexity 
of II • lljj and its dual norm. The additional requirement of convexity excludes the cases of 
nonconvex || • ||jj. For example, the case of 


N 


Ixlirt = llxll!! : = 


:= ^ |xi|^, 0 < p < 1 


2=1 


is not covered by Theorem 1. 


With very similar arguments, we can show the following theorem which characterizes 
the uniformly stable and robust reconstruction via the Dantzig type model by utilizing the 
(p, a)-robust width property. 


6 






Theorem 2. For any CS space {T-L,A, || • ||(t) with bound L and any linear operator ^ H ^ 
, the following are equivalent up to constants: 

(a) $ satisfies the (p, a)-robust width property over B^. 

(b) For every E FL, X > ^ ^ ^ satisfying ||$^a;||o < A, any solution x* to the 

following optimization model 


min ||x||(j, subject to + a;))||o < A 


{R> 


satisfies ||x* — x ^\\2 < — o|ltt + C*! • A for every a E A. 

In particular, (a) implies (b) with 
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V2p ^ 

provided P < Also, (b) implies (a) with 


p = 2Co, a = 


a^p 


K 


2rC'i’ 


where r = sup||^||^<i ||^>x|| 2 . 

Proof. The proof below follows from the pattern used for that of Theorem 1. Let z = x* —xf 
Step 1 : Since x* is a minimizer of ( |-RaD i h holds that ||x*||jj < ||x*’||jj. Now, repeat the 
argument for Step 2 in the proof of Theorem 1 to give 


< 2||a:^ -a|ls + 2L|lz||2. 


Step 2 : Prove the upper bound: 

||$z||^<2A||z||„. 


This follows from that 


||$'^$z||o < + '«;))||o + ||<h^u;|lo < 2A 

and 

\\<^z\\l = (z,<h^$z) < ||z||« • ||<h^$z|U. 

The remained proof of (a) ^ (6) follows by repeating the argument for Step 4 in the proof 
of Theorem 1. 

The proof of (b) (a). Pick such that ||<hx *’||2 < q:||x^|| 2 . Let A = ra||x ^||2 and 

to = —<f)x^ We have proved in the proof of Theorem 1 that such choosing of A and to 
satishes the constrained condition of < A and hence x* = 0 is the unique minimizer 

of (\Rx\ j. The remained proof of (6) ^ (a) follows by repeating the corresponding part in the 
proof of Theorem 1. □ 

Note that the convexity of || • ||(t is not involved in the proof of Theorem 2. 
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